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Distributed Source Coding with One Distortion 
Criterion and Correlated Messages 

Suhan Choi 
Abstract 

In this paper, distributed (or multiterminal) source coding with one distortion criterion and correlated 
messages is considered. This problem can be also called "Berger-Yeung problem with correlated mes- 
sages". It corresponds to the source coding part of the graph-based framework for transmission of a pair 
of correlated sources over the multiple-access channel (MAC) where one is lossless and the other is lossy. 
As a result, the achievable rate-distortion region for this problem is provided. It is an information-theoretic 
characterization of the rate of exponential growth (as a function of the number of source samples) of the 
size of the bipartite graphs which can represent a pair of correlated sources with satisfying one distortion 
criterion. A rigorous proof of the achievability and the converse part is given. It is also shown that there 
exists functional duality between Berger-Yeung problem with correlated messages and semi-deterministic 
broadcast channel with correlated messages. This means that the optimal encoder-decoder mappings for 
one problem become the optimal decoder-encoder mappings for the dual problem. In the duality setup, 
the correlation structure of the messages in the two dual problems, source distortion measure and channel 
cost measure are also specified. 

Index Terms 

Distributed source coding, correlated messages, Berger-Yeung problem, random graphs, duality. 

I. Introduction 

Consider a many-to-one communication scenario where many transmitters want to send correlated 
input sources reliably and simultaneously to one joint receiver over the channel. This scenario has many 
practical examples such as sensor networks and wireless cellular networks. For this transmission scenario, 
there have been two approaches. One is the joint source-charmel coding [1] and the other is the separate 
source-channel coding. The latter, which is based on Shannon's separation theorem in point-to-point 
communication, is divided into two multiterminal coding modules: the distributed source coding (DSC) 
[2], [3], [4] and the multiple-access channel (MAC) coding [5], [6]. 

In the DSC, many source encoders want to represent their correlated input sources into messages 
simultaneously and one joint source decoder reconstructs the original sources from the received messages 
(without channel errors) with satisfying certain distortion criterion. In the MAC coding, many channel 
encoders wish to send their input messages reliably and simultaneously over the channel, and one joint 
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channel decoder decodes the original messages from the channel output. Here, the messages are used as 
a discrete interface between source coding and channel coding modules. 

Before the recent introduction of graph-based framework [7], only independent messages are considered 
in this separation approach. However, it has been generally known that even though this separation ap- 
proach with independent messages gives convenient modularity, it is generally not optimal in multiterminal 
communications scenarios [1]. In the graph-based separation approach [7], correlated messages, which 
can be associated with bipartite graphs, are used as a discrete interface between source coding and chaimel 
coding modules. This approach enables us to maintain Shaimon-style modularity, and to minimize the 
performance loss as compared to the optimal joint source-chaimel coding [1]. 

More specifically, in the source coding module, the correlated sources are encoded (or represented) 
distributively into correlated messages. And these correlated messages are then encoded and reliably sent 
over the chaimel in the chaimel coding module. Here, the correlated messages correspond to edges in 
the graph. This graph-based framework was also applied to the broadcast chaimel [8]. 

For DSC with a pair of correlated sources (and independent messages), there have been three different 
problems in terms of distortion criterion: 

• Lossless DSC (Slepian-Wolf problem [2]) where both sources are lossless, 

• Lossy DSC (studied by Berger [3] and Tung [9]) where both sources are lossy, and 

• DSC with one distortion criterion (Berger- Yeung problem [4] where one source is lossless and the 
other is lossy. 

Note that Pradhan et al. [7] considered only lossless DSC with correlated messages, presenting the 
achievable rate region for this problem. For the lossy DSC with correlated messages, Choi [10] provided 
an inner bound to the achievable rate-distortion region. Through the result of [10], the graph-based 
separation framework in [7] can be extended to the transmission of analog correlated sources. This is 
because the same chaimel coding module in [7] can be used to send the correlated messages which are 
the outputs of both lossless and lossy source encoders. 

In this paper, we study a DSC problem with one distortion criterion and correlated messages. This 
problem can be also called "Berger- Yeung problem (BYP) with correlated messages". In this problem, 
two non-communicating encoders represent a pair of correlated sources into correlated messages and a 
joint decoder reconstruct the original sources, where the reconstruction of one source is lossless and that 
of the other is lossy with certain distortion criterion. In other words, this problem corresponds to the 
source coding part of the graph-based framework [7] for transmission of a pair of correlated sources over 
the MAC, where one is lossless and the other is lossy. 

As a result, we provide the achievable rate-distortion region for this problem, which is an information- 
theoretic characterization of the rate of exponential growth (as a function of the number of source samples) 
of the size of the bipartite graphs which can represent a pair of correlated sources with satisfying one 
distortion criterion for the transmission over the MAC. We present both the achievability and the converse 
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part of the proof. This indicates that a pair of correlated sources can be reliably represented into a nearly 
semi-regular bipartite graph even when one source is lossless and the other is lossy. 

It is also illustrated that a given pair of correlated sources can be efficiently represented into many 
different nearly semi-regular bipartite graphs without increasing redundancy. 

Therefore, based on the merged results of this paper, [7] and [10], it can be concluded that, for 
transmission of any (both discrete and continuous) set of correlated sources over the MAC, graphs can 
be used as discrete interface between source coding and channel coding modules. 

We also examine functional duality between our problem, Berger-Yeung problem with correlated mes- 
sages, and semi-deterministic broadcast channel (SBC) with correlated messages [8]. Similar researches 
have been presented in the literature. In [11], functional duality between Berger-Yeung problem and 
semi-deterministic broadcast channel was studied. Functional duality between distributed source coding 
and broadcast channel coding problems is also provided in [12]. However, it should be noted that 
previous studies [11] and [12] considered functional dualities in the case of independent messages only. 
Accordingly, it is natural to ask whether this duality holds for correlated messages or not. 

Consequently, we show that under certain conditions, for a given BYP with correlated messages, a 
functional dual SBC with correlated messages can be obtained, and vice versa. This means that the 
optimal encoder-decoder mappings for one problem become the optimal decoder-encoder mappings for 
the dual problem. We also specify the correlation structure of the messages in the two dual problems 
and source distortion measure and channel cost measure for this duality. 

The rest of this paper is organized as follows. We first give some preliminaries including the definition 
of bipartite graphs and the concept of correlated messages in Section JI] In Section |llll we formulate the 
problem and present the achievable rate-distortion region, which is one of the main results of this paper 
Thereafter, the proof of the theorem will be provided in Section |lVl Then, we discuss the representation 
of a correlated sources into many different graphs in Section |Vl In Section |VlJ we examine the functional 
duality between BYP with correlated messages and SBC with correlated messages. Finally, Section IVIII 
provides concluding remarks. 

II. Preliminaries 

Before we discuss the main problem, let us first define a bipartite graph and related mathematical 
terms which will be used in our discussion. 

Definition 1: • A bipartite graph G is defined as an ordered tuple G = {Vi{G),V2{G), E{G)) 
where Vi{G) and V2{G) denote the first and the second vertex sets of G, respectively, and E{G) 
denotes the edge set of G. i.e., E{G) C Vi{G) x V2{G). 

• If E{G) = Vi{G) X V2{G), then G is said to be complete. 

• The degree of a vertex u G Vi{G) in a graph G, deg^j i{u), is the number of vertices in V2{G) that 
are connected to u. Similarly deg(^ 2(^) defined for all v G V2{G). 

Since we use a special type of bipartite graphs, we define those bipartite graphs as follows. 
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Definition 2: A bipartite graph G is called nearly semi-regular with parameters (Ai, A2, A'^^, Ag, /x) 
for /X > 1, denoted by G(Ai, A2, A'^^, Ag, /x), if it satisfies: 

. = Aifor i = l,2, 

. Vu G Vi{G), A^^-i < degGr,i(n) < A'^fi, 

. Vt; G y2(G), A>-i < degG,2(^^) < A> 
where \A\ denotes the cardinality of a set A. 

Note that these nearly semi-regular graphs have slackness parameter fi for the degrees of the vertices. 
These nearly semi -regular graphs become semi -regular if /x = 1. 

In our problem, we also consider a pair of correlated messages (TVi,VF2) such that {Wi,W2) G 
Wi X Wi where two integer message sets Wi = {l,2,...,Ai} and W2 = {1,2, ...,A2} [7], [8]. 
We assume that there is some kind of correlatiorQ between two message sets. More specifically, if the 
messages Wi and W2 are independent, then all possible pairs {Wi, W2) in the set Wi x W2 are equally 
likely. On the other hand, if they are correlated, only some pairs {Wi, W2) G A such that A C Wi x W2 
are equally likely and the other pairs have zero probability. 

This correlation structure of the messages can be rephrased in terms of a bipartite graph G. The 
message pairs (W^i,VF2) G E{G) are equally likely with probability ]^^jyy> and the message pairs 
{Wi,W2) ^ E{G) have zero probability where a set E{G) C Wi x W2, and each individual message 
Wi and W2 are individually equally likely with probability and -pj^, respectively. If the messages 
are independent, E{G) = Wi x W2. 

Therefore, a pair of correlated messages (T^i, W2) is an ordered tuple (Wi, W2, G), which is charac- 
terized by two integer message sets Wi = {l,2,...,Ai} and yV2 = {1,2,...,A2}, and an associated 
bipartite graph W2, 
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Fig. 1. Independent and correlated messages 



Let us consider a simple example shown in [7], which is illustrated in Fig. [T] Here, Wi = W2 = 
{1,2,3}. The vertices in the bipartite graph denote messages, and edges connecting two vertices imply 

'Note that the meaning of correlation in the messages is different from the commonly used concept of correlation in the 
source coding problem. 
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that the corresponding message pair are equally likely. The complete bipartite graph of (a) corresponds to 
the independent messages where all the possible pairs have equal probability i. However, (b) shows the 
correlated messages where each message pair in {(1, 1), (1, 2), (2, 2), (2, 3), (3, 3), (3, 1)} has probability 
g, but (1, 3), (2, 1) and (3, 2) have zero probability. Moreover, (c) shows perfectly correlated messages 
where only three message pairs (1, 1), (2, 2) and (3, 3) can occur with the same probability ^. The 
messages of (c) have higher correlation than those of (b). 

III. Problem Formulation and Summary of Result 

In this section, we formulate the problem and show one of the main results of this paper Consider a 
pair of correlated sources Xi and X2 with a joint probability distribution p{xi,X2) and finite alphabets ^1 
and X2, respectively. In other words, a pair of correlated sources is an ordered tuple {Xi, Af2,p(xi, X2)). 
Let (Xii,X2i), (^i25-'^22)5 ... be a sequence of jointly distributed random variables i.i.d. ~ p{xi,X2). 
We denote Xf = {Xu, . . . , Xin) and X2 = {X21, . . . , ^2n)- We assume that the sources do not have a 
common part [13]. 

See the distributed source coding (DSC) system with correlated messages, shown in Fig. [2j where the 
inputs of encoders are two discrete memory less correlated sources Xi and X2. 
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Fig. 2. Distributed source coding with one distortion criterion and correlated messages 



The objective of this system is to represent the input (Xf ) into correlated messages (Wi, VF2) G 
Wi X W2 where Wj = {1,2, . . . , Aj} for i = 1,2, and to reconstruct the original sources from the 
received messages {Wi, W2) under certain distortion conditions. Here, the correlated messages {Wi, W2) 
can be associated with nearly semi-regular graphs G with parameters (Ai, A2, A'^, Ag, /i). Let Xi and 
X2 be reconstructions of Xi and X2, respectively, where Xi is lossless and X2 is lossy with certain 
distortion criterion. Hence, Xi = Xi and X2 may be different from X2. 

Definition 3: The distortion measures between {Xf,Xf) and ,^2) defined by 

1 " 

dxiiXi,Xi) = — > dx^{Xik,Xik), (1) 
n ^-^ 

k=l 
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1 

dxA^2^^2) = — / ^dx^iX2k,X2k), (2) 
k=l 

respectively, where the distortion measures d^^ and are any functions such that dx^ : X\^X\ ^ {0, 1} 
and d^^ : X2 x X2 ^ M+. Let dx,{xi,xi) = 1 - da:,xi where 6^^^^ = 1 if xi = xi and ^j;,^^ = if 

Xl / Xl. 

A similar multiterminal source coding problem with one distortion criterion was studied by Berger 
and Yeung [4]. So, this problem is also called the Berger-Yeung problem. However, note that they only 
considered independent messages as outputs of encoders. In our problem, we consider correlated messages 
where the correlation structure is captured by a bipartite graph as illustrated in Section |lll Hence, we 
also refer to our problem as "Berger-Yeung problem with correlated messages". 

Now, let us define a DSC system with one distortion criterion and correlated messages as follows. 

Definition 4: An (n, Tx-, , r^^J-DSC system for a pair of correlated sources (Xi, X2) and a nearly semi- 
regular bipartite graph G(Ai, A2, A'^^, Ag, /u) is an ordered tuple (/i,/2,(?), consisting of two encoding 
mappings /i and /2, and one decoding mapping g: 

, h ■■ ^ Vi{G), /2 : ^ V2{G), 
. g : E{G) ^ XJ" x X^, 

• such that a performance measure is given by the average distortions = E^^dx^iX^, Xf) and 

rx,=ExMX^^,X^) where (X^, X^) = g{h{X^^), h{X^^)). 
We also define achievable rate-distortion tuples for this problem as follows. 

Definition 5: A rate-distortion tuple {Ri, R2, R[, R'2, D) is said to be achievable for a pair of 
correlated sources {Xi, X2,p{xi,X2)), if for any e > 0, and for all sufficiently large n, there exists a 
bipartite graph G(Ai, A2, A'^, A2, /x) and an associated (n, Ta;^, XjjJ-DSC system as defined above such 
that: i log Aj < i?j + e for i = 1, 2, ^ log A^ < i?^ + e for i = 1, 2, ^ log fi < e and the corresponding 
average distortions t^^ < e and Tx^ < D + e. 

The goal is to find the achievable rate-distortion region TZ{V) which is the set of all achievable rate- 
distortion tuples [Ri, R2, R'l, R2, D). We have found an information-theoretic characterization of 7i{V), 
which is one of the main results of this paper. 

Theorem 1: 71{V)* = 7i{V) where 

n{Vr= U {{R^,R2,R!^,R!2,D): 

p{v\xi,X2) 

Ri>Ri>0 for i = 1,2, (3) 

R[>H{Xi\V), (4) 

R'2>I{V;X2\Xi), (5) 

Ri + R'2 = R[ + i?2 > ^(^1) + I{V; X2\Xi)} (6) 
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where V is (i) an auxiliary random variable with finite alphabet V satisfying |V| < |<^2| + 2, and 
p{xi,X2,v) = p{xi, X2)p{v\x2) forms the Markov chain Xi ^ X2 V, and (ii) there exists X2(Xi, V) 
such that E^^d^^{X2,X2) < D. 

Remark 1: Note that if we choose V = X2, then we get Theorem 3 in [7], that is the achievable 
rate region for lossless distributed source coding for a pair of correlated sources using graphs, i.e., 
Da:, = = 0, given by 

Ri>Ri>0 for i = 1,2, (7) 

R[>HiXi\X2), (8) 

R'^>H{X2\Xi), (9) 

Ri + R'2 = R[ + R2>H{Xi,X2). (10) 
Remark 2: Note also that there is a close connection between lossy distributed source coding problem 
using graphs [10] and this problem. In [10] where both sources are lossy, an inner bound to the achievable 
rate-distortion region is given by: 

[J {{Rl, R2,Rl, R2t Da:,, Da:^) : 
p(u,v\xi,X2) 

Ri>Rt>0 for i = 1,2, (11) 

R[>I{U;Xi\V), (12) 

R'2>I{V;X2\U), (13) 

Ri + R'2 = R[ + R2>IiXi,X2;U,V)} (14) 

where U and V are (1) auxiliary random variables with finite alphabets U and V, respectively, and 
p{xi, X2,u,v) = p{xi, X2)p{u\xi)p{v\x2) fomis the Markov chain U Xi X2 — > V, and (2) there 
exist Xi {U, V) and X2{U, V) such that -E^r.^x; {Xi,Xi) < D^:, for i = 1,2. If we choose U = Xi'm the 
Theorem 1 in [10], which is shown above, then we can obtain Theorem [T] 

Remark 3: Theorem [T] gives only a partial characterization of the set of all nearly semi-regular bipartite 
graphs which can represent the given pair of correlated sources with certain amount of distortion. 

IV. Proof OF Theorem [T] 

In this section, we present the proof of Theorem [T] The proof consists of two parts: (1) the achievability 
of TZiV)* showing 7^(D)* C 71(1)) and (2) the converse part showing 7^(D)* D n{V). 

A. Achievability of TZ{'D)* 

The proof of this achievability is similar to that of [10, Theorem 1]. We use the random binning 
technique used by Berger [3], the concept of super-bin [8], and the notion of strongly jointly typical 
sequences. 
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Given a pair of correlated sources {Xi,X2) with distribution p{xi,X2), consider an auxiliary random 
variable V which satisfies the conditions (i) and (ii) in Theorem [T] Let us consider a fixed distribution 

p{xi,X2,v) = p{xi, X2)p{v\x2). Also, fix £ > 0, and an integer n > 1. Let us choose {Ri, R2, R'l, R'2) 
as follows. Ri > H{Xi) - I{Xi; V) +e, R2> I{V; X2) - I{Xi;V) + e, and iii + ii^ = R[ + R2 = A 
where A = H{Xi) + I{V; X2) - I{Xi;V) + 2e. 

Codebook Generation: First, draw 2"(^(^i)+^) n-length sequences Xf{k), for A; € {1, 2, ... , 2"(^(^i)+^)}, 
independently from Ai"'^ {Xi ) with probability r-nr^j-. — rr where A^^^ {Xi ) is the strongly e-typical set with 
respect to the distribution p{xi), which is the marginal of p{xi,X2, v). Call this collection Ci. Similarly, 
generate 2"(^(^'^^)+^) sequences ^{1), for / e {1, 2, . . . , 2"(^(^'^^)+^)}, from Ai^'^V), and call this 
collection C2. Then, divide Ci into 2"^^ equal-size bins B{i) for i G {1, 2, . . . , 2"^^ }. Similarly, generate 
C(j) for j G {1,2. . . ,2"^==} from C2. This step is illustrated in Fig. [3l where solid lines in bins B{i) 
and C(j) denote n-length sequences Xf and F", respectively. 

Graph Generation: As shown in Fig. [3l a random bipartite graph G can be generated from the bin 
indices of codebooks Ci and C2 and jointly typicality as follows. (1) Vi{G) = {1, 2, . . . , 2"^^ } and 
V2{G) = {1, 2, . . . , 2"-f^^}, (2) V(i,i) G Vi{G) x V2{G), G E{G) if and only if there exists at least 
one e-strongly jointly typical sequence pair in B{i) x C{j). 




Fig. 3. A bipartite graph G(2"^i, 2"^^ 2"^'i, 2"^^, fj.) generated from random codebooks Ci and C2. 

Encoding Error Events Due to the Degree Condition: Before the encoding and decoding steps, let us 
make sure that the generated graph G satisfies certain requirements. If the vertices of G do not satisfy 
this degree requirements, the sources may not be able to be reliably represented by using this graph. So, 
an encoding error will be declared if either one of the following events occur: 

. El. 3i G Vi{G) such that | Mogdeg^j ^(i) - R'2\ > e', 
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. E2: 3j G V2{G) such that |i logdegc.sO') - > e', 
where ei(e) is a continuous function of e with ei(e) ^ as e — > 0, and is characterized in Appendix, 
and e' > 3ei. 

Choosing Message Correlation: If none of the above error events Ei and E2 occurs, then choose G = G. 
If any of the above two error events occurs, then pick any nearly semi-regular graph with parameters 
(2"^i,2"^2,2"^'i,2'^^^2"'^') and call it G and no guarantee will be given regarding the probability of 
decoding error. 

For this graph G, and the given correlated sources Xi and X2, using the above random codebooks Ci 
and C2, we construct an (n, , r^^J-DSC system, where r^^ < e and t^^ < D + e. 
Encoding: If (£'1 U E2) does not occur, for the given source X", encoder 1 looks for a sequence 
x^{k) € Ci such that = , and sends the bin index i such that G B{i). If there is no such 

index i, it sends any random index i chosen uniformly from {1, 2, ... , 2"^^}. 

For the given source X2, encoder 2 looks for a sequence € C2 such that {X2,v^{l)) G 

A'f'\X2,V) where Ai"'\X2,V) is the jointly strongly e-typical set with respect to the distribution 
p{x2,v), and sends j satisfying G C{j). Let us denote the selected sequences by v'"'{X'2). 

Then, from the Markov lemma [3], [14], (Xl^X^, f"(XJ)) becomes jointly typical, i.e., {X'^ , X^ , v"- (X^)) G 
A^p\Xi, X2, V) where e = Ke for an appropriate constant K. If there is no such index j, it sends any 
random index j chosen uniformly from {1, 2, ... , 2^^^}. 

Decoding: Given the received index pair the decoder looks for the unique pair of sequences 

{X^,v'^{X'^)) G B{i) X C(j) such that {X^,v'^{X^)) G aP{Xi,V). Then, it calculates Xf' and 
from Xim = Xim and X2m = X2(Xim, fm(X2 )) for 1 < m < n. 

If there exists any other (x^,f") G B{i) x C{j) such that / (X^", ^"(Xf )) and (a;^,t;") G 

4''^(Xi,F), then an error is declared, and it sets X" = x" and X2 = X2 where and arbitrary 
sequences in Af" and A'^', respectively. 

Probability of Error Analysis: Let E denote the error event. Then, the probability of error P{E) can 
be given by 

P{E) = P{EiUE2)P{E\EiUE2)+P{EnE'lnE^) (15) 
<P{EiUE2)+P{Er\E'^r\E^). (16) 

By using the similar techniques shown in [8, p. 2847], it is can be shown that, for sufficiently large 
n, P{Ei) < I, and ^(£'2) < f , if -R2 = ^ - -Ri and R[ = A - R2, respectively. This means that with 
high probability we can obtain a nearly semi-regular bipartite graph G such that each vertex in Vi{G) 
has degree nearly equal to 2"^^ and each vertex in V2 (G) has degree nearly equal to 2"^i . 

The second probability in ( fT6l ) can be bounded as given in the following lemma. 

Lemma 1: For any e > 0, and sufficiently large n, 

P{E n El n E^2) < V (17) 
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Proof: Now let us calculate the probability P{E n E'f n E2). If previous error events Ei or E2 do not 
occur, we define other error events as follows: 

i?3 : {X^,Xl^)iAt\XuX2), 
E4 : $Xf G Ci, 

^5 : h^'iX'd) e C2, 

Eg : (Xf,X2",t;"(X2")) ^4")(Xi,X2,F), 

^7 : 3{x1, t;") / (Xf , such that t;") € B{i) x and (x^ X^, v'^) G X2, F). 

Then, 

PiE n El n E^) = p {uU^Ei n El n E^) (18) 

7 

< J^P(s,nsfn^|) (19) 

i=3 

By the property of jointly typical sequences [14], P(i?3 H Ef n E2) < f for sufficiently large n. 
P{Ei n n < f since |Ci| > T'^^^^^ and from the property of typical set [3, p. 371]. ^(£^5 n 
El n < f since IC2I > 2"^^^^^^) [3, Lemma 2.1.3]. P{Eii r\Elr\E^) <^ from the Markov lemma 
as described in [3], [14]. P{E^ <r^El<r^E^) < ^ since i?i + -R2 > (^i) + I{V; X2) - I{Xi]V) + 2e 
[3]. 

Therefore, by applying the union bound we have P{E) < P{Ei) + P{E2) + P{E r\Elr\ E^) < e. 
Calculation of Distortion: Now let us calculate the resulting distortion d^^iXJ^ , X"^)- Following [3], if 
the error event E does not occur, dx^ {X^;, X2) < D + e* from the jointly typicality of (Xf, , ^"(Xg )) 
where e* — > as e ^ 0. So, 

dxAX^,X!i) < (1 - P{E)){D + e*) + P{E)dm.. (20) 

where dmax is the maximum distortion for any individual sequence. 

Hence, for sufficiently large n, the distortions for Xf and X2 can be close to and D, respectively, 
if P{E) is small. 

In every realization of random codebooks, we have obtained a graph G with parameters (2"^^ 2"^^ ,2"^'i ,2"^^ ^2"'^' ), 
and averaged over the ensemble of random codebooks, the average distortions for Xf and X2 are close 
to and D, respectively. Therefore, the proof of the achievability is complete. ■ 

B. The Converse 

Now we prove the converse part of Theorem [T] Some steps of the proof is similar to the converse 
proof in [4]. 

Let us assume a rate-distortion tuple {Ri, R2, R'l, R'2, D) is achievable. Then for any e > 0, and for all 
sufficiently large n, there exists a bipartite graph G(Ai, A2, A'^^, A2, ;u) and an associated (n, Ta;^ , r^^J- 
DSC system as defined in Definition |4] such that: ^ log Aj < i?j + e for i = 1, 2, ^ log A ■ < i?- + e 
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for f = 1,2, ilog/i < e and the corresponding average distortions r^^ < e and t^^ < D + e. Let 
fi{Xf) = Wi and f2{XS) = W2. 

Then, from the vector version of Fano's inequality [4, Lemma 1], we can have 

HiX^lX"^) < nr^, \og{\Xi\ - 1) + n/?6(r,J < ne„ (21) 

where e„ ^ as n — > cxo and -ffb(-) is the binary entropy function. 

Using the constraints on the degree of the vertices in the nearly semi-regular graph associated with 
correlated messages, we have 

ni?; > H{Wi I W2) - log /u (22) 

>I{Wl■X'^\W2)-\og^i (23) 

= H{X'l\W2) - H{X'i\Wu W2) - log/u (24) 

H{X'l\W2) - H{Xl\Wi,W2,X'^) - log^ (25) 

(b) " 

> Y.HiXikm^X'i-^) - H{X^\X^) - \ogii (26) 

k=l 
n 

> H{Xik\W2,X^-\X^k+i),- ■ ■ , Xin) - H{X^\X^) - logfi (27) 

k=l 

(c) " 

> Y,H{X^k\Vk)- nen -log (28) 
fc=i 

where (a) is from J(X{^; = due to the Markov chain ^ (Wi,W^2) ^ Xf, (b) is 

obtained by using the chain rule and removing conditioning, and (c) follows from Fano's inequality (|2TI ) 
and by defining Vk = {W2, X'[-\ X^f,+l), ■ ■ ■ , ^in)- 
Thus, we have 



n 

R[ >-y^H{X^k\Vk)-en--logii. 

k=l 



(29) 



Similarly, we also have 

nR'2 > H{W2\Wi) - log (30) 

= H{W2)- IiWi;W2)-logn (31) 

> HiW2) - liX^; W2) - log /i (32) 

= H{W2\X'l)-\ogii (33) 

>/(X2";H^2m-log^ (34) 

= H{X1^\X^) - H{X1^\X^, W2) - log/i (35) 
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^[H{X2k\Xik) - H{X2k\W2,Xl\Xt^)] - log/i (36) 

k=l 

> ^[H{X2k\X^k) - H{X2k\W2,X^)] - logfi (37) 

k=l 



id) 

k=l 



^[HiX2k\Xik) - H{X2k\Vk,X^k)] - log^ (38) 



= Y,HVk;X2k\Xik)-logfi (39) 

k=l 

where (a) is from I{Wi; W2) < H^i'-: ^2) due to the Markov chain W2 ^ X^ ^ Wi, (b) is obtained by 
using the chain rule and memory less property of the sources (Xi, X2), (c) is from removing conditioning, 
and (d) is obtained by defining Vk = (W2, -'^i"^, -^^1(^+1), • • • , Xin). 
So, we have 

1 " 1 
R'2 >-y^I{Vk;X2k\X,k) - -log/i. (40) 

k=l 

For the sum-rate Ri + R2 = R'l + R2, using the constraints on the size of the edge set and on 
the degree of the vertices of the nearly semi -regular graph associated with correlated messages, i.e., 

{Wi,W2) G E{G) and l£;(G)| < ^2<^^+^'^\ we have 

n{Ri + R'2) = n{R[ + R2) 

>H{Wi,W2)-log^i (41) 

= H{Wi) + H{W2\Wi) - log/i (42) 

= I{X^- Wi) + H{Wi\X^) +H{W2\Wi) - log/x (43) 
=0 

H{X^) - H{X^\Wi) + H{W2\Wi) - log/x (44) 

(b) 

> H{X'{) - nen + H{W2\Wi) - log^ (45) 

n 

Y}H{X^k) + I{Vk;X2k\X^k)\ - nen - log/x (46) 
fc=i 

where (a) follows the fact that H{Wi\X'l) = since = Wi, (b) is obtained from H{X'l\Wi) < 

nen due to the Fano's inequality [14], (c) follows from the chain rule and the memoryless property of 
the source Xi and ^ and ^ above, i.e., H{W2\Wi) > ELi ^(^fe! ^2fc|^ifc)- 
Hence, we have 

Ri + R'2 = R'l + R2 (47) 

1 " 1 

>-S2[H{Xik)+I{Vk;X2k\Xik)]-en--logfi. (48) 

k=i 

Therefore, we can have the converse by taking the limit of inequalities (l29l ). (l40b and (l48l ) as n — > 00. ■ 
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V. Representation of a Pair of Correlated Sources into Different Graphs 

In this section, we show that in the case of DSC with one distortion criterion, a pair correlated 
sources {Xi,X2) can be reliably represented into many different graphs without increasing the redun- 
dancy. In other words, if (i?i,i?2) are inside the triangle ACD in Fig. |4] and R'l and i?2 satisfy the 
condition Ri + R'2 = R'l + R2 = H{Xi) + I{V; X2\Xi), then many different graphs G with parameters 
(-2ni?i 2"^22 2"'Ri,2"^2^2"''') can represent the same sources {Xi,X2). Note that this is similar to the 
lossless DSC case shown in [7, Section 5]. 



R 2 i 




H(X,\V) H(X,) 
Fig. 4. The achievable rate-distortion region of DSC with one distortion criterion in terms of Ri and R2 

Let us consider some special cases as follows. 

. Point A: Ri = R[ = H{Xi), R2 = R'2 = I{V;X2\Xi) = I{V;X2) - I{Xi;V). In this case, we 
get a nearly complete graph, which is an efficient representation of the sources that has the least 
redundancy in the conventional sense, i.e., in terms of R2). For this point, the bin sizes \B{i)\ 
and |C(j)| in Fig. |3] are roughly unity and 2^^'^'^'^''^\ respectively. 

. Point B: Ri = R[ = H{Xi) - a, i?2 = R2 = ^2) -/(^i; F) + a for < a < I{Xi;V). This 
case corresponds to an arbitrary point (i?i,i?2) on the line segment AC. This also gives a nearly 
complete graph, where \B{i)\ and in Fig. [3] are roughly 2"" and 2"(^("'*^i'^)~"), respectively. 

. Point C:i?i = R[ = H{Xi\V) = H{Xi)-I{Xr,V), R2 = R'2 = /(F; X2). This also gives a nearly 
complete graph, where \B{i)\ and \C{j)\ in Fig. [3] are roughly 2"^("^i'^) and unity, respectively. 

. ^omtD:R^ = H{X{),R2 = I{V;X2),R'^ = H{Xi\V) = H{Xi)-I{Xi-V),R'2 = I{V;X2\Xi) = 
X2) — I{Xi; V). In this case, the graph has the maximum redundancy in the conventional sense, 
and is not complete. However, this is also an efficient representation since the total number of edges 
of the graph is nearly equal to 2"(^(^i)+^(^'^^)~^('^i'^)), where \B{i)\ and \C{j)\ in Fig. [3] are 
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roughly unity and unity, respectively. 

Therefore, an arbitrary point (i?i,i?2) on the line segment AC (such as point A, B and C) has 
the least redundancy and point D has the maximum redundancy in the conventional sense. For every 
point in the triangle ACD in FigjH we can obtain an equally efficient representation of the correlated 
sources {Xi,X2) into a nearly semi-regular graph. Here, equally efficient representation means that the 
cardinalities of edge sets of these graphs for the different values of {Ri, R2) are nearly the same. 

VI. Functional Duality Between Berger-Yeung Problem with Correlated Messages 
AND Semi-deterministic Broadcast Channel with Correlated Messages 

In this section, we discuss functional duality between Berger-Yeung problem (BYP) with correlated 
messages and semideterministic broadcast channel (SBC) with correlated messages. We show that, under 
certain conditions, for a given BYP with correlated messages problem, a dual SBC with correlated 
messages problem can be obtained where both problems have the same joint distribution and the same 
correlation structure in the messages, and vice versa. Before discussing the functional duality, we briefly 
recall BYP with correlated messages in Section |lll] and SBC with correlated messages in [8, p. 2848]. In 
our discussion, we denote the given distribution and the distribution which optimizes a given objective 
function by p{-) and p*{-), respectively. 



A. Berger-Yeung Problem with Correlated Messages 
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Encoder 2 
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Channel 
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1 1 

3 3 



Fig. 5. Berger-Yeung problem with correlated messages 



Consider a BYP with correlated messages, shown in Fig. \5\ where Xi and X2 are two correlated 
discrete memoryless stationary sources with a given joint probability distribution p{xi,X2), and with 
finite alphabets Afi and X2, respectively. The objective of this system is to represent {Xf,X2) into 
correlated messages {Wi, W2) € Wi x W2 such that Wj = {1, 2, ... , Aj} for i = 1, 2, which can be 
associated with nearly semi-regular graphs G(Ai, A2, A'^, Ag, /u) and to reconstruct the original sources, 
(X^jXg), from the graphs under certain distortion conditions. Here, the encoders do not communicate 
with each other. 
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Let d : Xi X X2 X X2 —>■ be the joint distortion measure, where X2 is the reconstruction alphabet of 
X2. The encoders are given by Vi{G) for i = 1,2, and the decoder is given by E{G) — > <Yf x 

An achievable rate-distortion region TI{D)byp for a distortion constraint D is given by 

n{D)BYP= U {{Rl,R2,R'i,R2,D): 

p{v\xi,X2) 

Ri>Ri>0 for i = 1,2, (49) 

R[>H{Xi\V), (50) 

R'2>I{V;X2\Xi), (51) 

iii+i?^ = R[ + R2> H{Xi) + /(y;X2|Xi)} (52) 

where V is an auxiliary random variable with finite alphabets V satisfying |V| < |^2|+2, a.ndp{xi,X2,v) = 
p{xi, X2)p{v\x2) forms the Markov chain Xi ^ X2 ^ V , and there exists X2{Xi, V) such that Ed < D 

and X2^V ^ iXi,X2). 

Note that //(Xi I F) = H{Xi)-I{Xi;V) and I{V; X2\Xi) = I{V; X2)-I{Xr,V) due to the Markov 
chain ^1^X2^ V. So, the sum-rate R'^g^p{D) = Ri + R'2 = R'l + R2 can be also expressed as 

i?|jT(Z)) = min[F(Xi) + I{V; X2) - I{Xr,V)] (53) 

where the minimization is taken over ^(^1x2) and p{y2\v,xi)- 



B. Semideterministic Broadcast Channel with Correlated Messages [8] 
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Fig. 6. Semideterministic broadcast channel with correlated messages 



Consider a general discrete memoryless stationary SBC system with correlated messages, shown in 
Fig. m with a given conditional distribution p{x2\x) and a deterministic function xi = f{x) where X 
is input alphabet and Xi are output alphabets, respectively, for i = 1,2. The objective of this system is 
to send simultaneously a pair of correlated messages (Wi,VF2), which can be associated with graphs 
(S'(Ai, A2, A']^, A2, /u), to the two receivers over the channel where (M^i,VF2) £ VVi x W2 such that 
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Wi = {1,2,..., Aj} for i = 1,2. Here, the decoders do not communicate with each other. We assume 
that there is no common message in the two messages. 

Let w : —^ be the input cost measure associated with this channel. The encoder is a mapping 
E{G) ;f and the decoders are given by A'/^ Vi{G) for i = 1,2. 

The capacity region 'JZ{W)sbc for a cost constraint W is given by 

n{W)sBc = U {(Ri,R2,R'i,R2) ■■ 

p(v,x) 

Ri>Ri>0 for i = 1,2, (54) 

Ri < H{Xi), (55) 

R2<I{V;X2), (56) 

Ri+R!2 = R!^+R2 < H{Xi\V)+I{V;X2)} (57) 

such that Ew < W, V ^ X ^ {Xi,X2) where V is an auxiUary random variable with finite alphabet 
V satisfying |V| < \X\ + 2, and p{v,x) = p{v)p{x\v) forms the Markov chain X2 ^ V —> X. 
The sum-rate = R^ + R'^ = R[ + R2 is given by 

RTSc'iW)=me,x[H{Xi)+I{V;X2)-IiXr,V)] (58) 

where the maximization is taken over p{v,x). 

The achievable rate-distortion region of BYP and the capacity region of SBC with independent and 
with correlated messages are depicted in Fig. |2l where R'^ = Ri — a for i = 1,2 such that < a < 
min{i?i, i?2}- Note that a = for independent messages and a > for correlated messages. 

C. Functional Duality between BYP and SBC with Correlated Messages 

Now, we discuss functional duality between BYP with correlated messages and SBC with correlated 
messages. We show that, under certain conditions, for a given BYP with correlated messages, a dual SBC 
with correlated messages can be obtained where both problems have the same joint distribution and the 
same correlation structure in the messages, and vice versa. 

The following theorem is one of the main results of this paper. 

Theorem 2: (1) For a given BYP with correlated messages {Wi, W2), which can be associated with 
a graph G(Ai, A2, A'^^, Ag, /i), a given source {Xi,X2) ~ p{xi,X2) with alphabets Xi for i = 1,2 and 
reconstruction alphabet X2, a distortion measure d : XiX X2X X2 ^ M^", and a distortion constraint D, 
suppose {p* {v\x2),p* {x2\xi,v)} achieves the minimum of the sum-rate distortion function R^j^^p{D): 

R'^^p{D)=m.m[H{Xi)+I{V;X2)-I{Xi;V)] (59) 

such that < D,Xi ^ X2 andX2 -^V ^ {Xi,X2)- Then,p(xi,X2) and {p* {v\x2) ,p* {x2\xi,v 
induce the following joint distribution: 

P*{XI,X2,V,X2) = P{XI,X2)P*{V\X2)P*{X2\XI,V). (60) 
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Fig. 7. Achievable rate-distortion region of BYP and the capacity region of SBC with independent and correlated 
messages 



Let p*{v), p*{x-i,X2\v) and p*(a;2|xi, £2) be the corresponding marginals. If p*{xi,X2.,v,X2) satisfies 
V ^ X ^ {Xi,X2) where X = {Xi,X2), then 3 a dual SBC problem for the channel p*{x2\x) = 
p*{x2\xi,X2) and xi = f{x) with correlated messages {Wi, W2), which can be associated with a graph 
G(Ai, A2, A'^, A2, /u), input alphabet X and output alphabets Xi, for i = 1,2, a cost measure w : X ^ 
M"*", and a cost constraint W such that: 

. R'^{l'p{D) = Rf£hiW), i.e., mm[H{Xi)+I{V;X2)-I{Xi;V)] = maic[H{Xi) + I{V; X2) - 
I{Xi; V)] where the minimum is taken over p{v\x2) and p{x2\xi, v) with the given source distribu- 
tion p{xi,X2) such that Ed < D, Xi ^ X2 V and X2 ^ V ^ (Xi, X2); and the maximum is 
taken over p{v) and p{x\v) with the fixed channel conditional distribution p*{x2\x) and xi = f{x) 
such that Ew <W,V X ^ (^1,^2), and X2^V -> X, 

• the distributions p*{v) and p*{x\v) = p*{xi,X2\v) obtained from the BYP achieve the maximum in 
the dual SBC problem, 

• the correlation structure of the messages (^^1,^2) of the dual SBC problem is the same as that 
of the given BYP, i.e, the graph G(Ai, A2, A^, A2, /x) of the dual SBC is the same as that of the 
given BYP 

provided the cost measure and the cost constraint are chosen such that 

w{x) = ciD{p* {xi, X2\x)\\p{xi, X2)) +0 (61) 
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and W = £'p.(j.) where -D(-||-) is the relative entropy [14], ci > and 9 are arbitrary constants. 

(2) For a given SBC with correlated messages {Wi,W2), which can be associated with a graph 
G(Ai, A2, A[, A2, /u), and a given channel conditional distribution p{x2\x) and xi = f{x), input alphabet 

and output alphabets for i = 1,2, a cost measure w : X ^ and a cost constraint W, suppose 
{p* {v) , p* {x\v)} achieves the maximum of the sum-rate cost function Rsbc(W): 

RT£ciW)=max[H{X^)+I{V;X2)-I{Xr,V)] (62) 

such that Ew <W, V ^ X ^ {Xi,X2), and X2 ^ V ^ X. Then, p{x2\x) and {p* {v) , p* {x\v)} and 
X = {Xi,X2) induce the following joint distribution: 

P*{v,Xl,X2,X2) = p*{v)p*{xi,X2\v)p{x2\xi,X2). (63) 

Let p*{xi,X2), p*{v\x2) and p*{x2\xi,v) be the corresponding marginals. If p*{v,xi,X2,X2) satisfies 
Xi — > X2 V, then 3 a dual BYP with correlated messages {Wi, W2), which can be associated with 
a graph G(Ai, A2, A'^, A'2, fi), for the source (Xi, X2) ~ p*{xi, X2) with alphabets Xi for i = 1, 2 and 
a reconstruction alphabet X2, a distortion measure d : Xi x X2 x X2 ^ M+, and a distortion constraint 
D such that: 

• ^IscW = RBYpiD), i.e., max[i/(Xi)+/(y;X2)-/(Xi;y)] = uAn[H {X^) + 1 {V ; X2) - 
I{Xi;V)\ where the maximum is taken over p{v) and p{x\v) with the fixed channel conditional 
distribution p{x2\x) and xi = f{x) such that Ew < W, V ^ X ^ {Xi,X2), and X2 ^ V ^ X; 
and the minimum is taken over p{v\x2), p{x2\xi,v) and the fixed source distribution p*{xi, X2) such 
that Ed<D,Xi^X2-'V and X2 ^ V ^ (Xi, X2); 

• the distributions p*{v\x2) and p*{x2\xi,v) induced from the SBC problem achieve the minimum in 
the dual BYP, 

• the correlation structure of the messages {Wi, W2) of the dual BYP is the same as that of the given 
SBC problem, i.e, the graph G(Ai, A2, A'^, Ag, /i) of the dual BYP is the same as that of the given 
SBC 

provided the distortion measure and the distortion constraint are chosen such that 

d{xi,X2,X2) = -C2 \ogp{x2\x)+dQ{xi,X2) (64) 

and D = Ep.i^^^^^^)p.(^i.^\^^^^^)[d{Xi,X2,X2)] where C2 > and (io(xi,X2) are arbitrary. 

Proof: Theorem |2] can be proved by applying the similar technique used in the proof of Theorem 1 
in [12]. For the sake of brevity, we omit the redundant part of the proof. The different part of the proof 
is to show that the correlation structure of the messages in the dual SBC problem is the same as that of 
the given BYP, and vice versa. 

For the given BYP with correlated messages, the correlation structure is determined by the random 
bipartite graph G generated from the bin indices of the random codebooks Ci and C2 and the jointly 
typicality of auxiliary random variables Xi and V as shown in Section IIV-AI The correlation structure 
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of the dual SBC problem with correlated messages is also determined in a similar way which is shown 
in the proof of Theorem 1 in [8]. More precisely, if we choose U = Xi and assume that there is no 
common message Wq in the proof of Theorem 1 in [8], then the correlation structure of the dual SBC 
problem with correlated messages is also determined by the random graph G which is generated from 
the bin indices of the random codebooks and the jointly typicality of auxiliary random variables Xi and 
V. Note that the same auxiliary random variables Xi and V are used for both BYP and its dual SBC. 
Therefore, it is obvious that the correlation structure of the messages {Wi, W2) of the dual SBC problem 
can be the same as that of the given BYP if the same random codebooks are used for both cases. ■ 
Remark 4: Theorem |2] is similar to Lemma 4 in [11], presenting the functional duality between BYP 
and SBC with independent messages. Theorem |2] above is also analogous to the Theorem 1 in [12], 
showing the functional duality between distributed source coding and broadcast channel coding with 
independent messages. 

However, note that there are significant differences as follows; 1) Lemma 4 in [11] and Theorem 1 in 
[12] show the functional duality in the case of independent messages only, whereas Theorem |2] above 
extends the functional duality to the case of correlated messages. 2) Moreover, Theorem |2] specifies the 
correlation structure of the messages in the two dual problems, while there is no such consideration in 
[11] and [12]. 

VIL Conclusion 

We have considered a distributed (or multiterminal) source coding problem where two non-communicating 
encoders represent a pair of correlated sources (for transmission over multiple-access channels) into 
correlated messages, which can be associated with an undirected nearly semi-regular bipartite graph, and 
a joint decoder reconstruct the original sources. Here, the reconstruction of one source is lossless and 
that of the other is lossy with certain distortion criterion. As a result, we have shown that the given 
correlated sources can be represented into such graphs with satisfying distortion criterion, providing 
the information-theoretic achievable rate-distortion region for this problem. Therefore, by merging the 
results of this paper, [7] and [10], we can conclude that a nearly semi-regular bipartite graph can be 
used as discrete interface in Shannon-style modular approach for transmission of any (either discrete or 
continuous) set of correlated sources over the multiple-access channels. 

We have also shown that under certain conditions there exists functional duality between our problem, 
"Berger-Yeung problem with correlated messages", and semi-deterministic broadcast channel with cor- 
related messages. We have also specified the correlation structure of two dual problems and the source 
distortion measure and the channel cost measures for the duality. 

Appendix 

A characterization of ei{e): For a precise characterization of the error events, we need a function of e, 
and certain properties of typical sets. For any pair ([/, V) of finite-valued random variables, there exists 
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[14], [15] a continuous positive function ei(e) such that (a) ei(e) ^ as e ^ and (b) for all e > 
(sufficiently small), there exists an integer A^o(e) > such that Vn > No{e) the following conditions 
hold simultaneously: 



2n{H{U)-ei) <|^(n)([/)| < 2'^(H(U)+ei) ^ 
2n{H{V)~€^) <|^W(y)| < 2"(-f^(^)+«i), 

2n{H{u,v)-€^) <|^W({7^y)| < 2^(H(uy)+^^) ^ 



-log 



n 



I{U-V) 



< 3ei. 



(65) 
(66) 
(67) 

(68) 



P{(w",F") G Ai"^\u,V)} 
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